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REMARKS 

Claims 46, 48, 50-54 and 56, 58-62 are currently pending in this 
application. Claims 46 and 54 have been amended and claims 47, 49, 55 and 57 
have been cancelled. No new matter has been added to this application. 

Rejection of Claims 46, 48, 52-54, 56 and 60 under 35 U.S.C. § 102 (e) 

The Examiner has rejected claims 46, 48, 52-54, 56 and 60 under 35 
U.S.C. § 102 (e) as being anticipated by U.S. Patent No. 6,898,307 (Harrington). 
The Examiner contends that Harrington teaches Applicant's invention as claimed. 
Applicant respectfully traverses the rejection. 

The present invention is directed to a method for utilizing augmented 
reality (AR) technology to combine real and virtual worlds together to provide an 
interface for a user to sense and interact with virtual objects in the real world. 
Video data comprising images of a moving model plane having markers is 
provided. The markers calibrate a camera to track the motion of the model 
plane. A pose of the moving model plane is determined according to the markers 
in the video data and calibration results. A three dimensional image data model 
of a product is provided in a pose corresponding to the pose of the moving model 
plane. An image correspondence between the camera calibration results 
associated with the moving model plane and the three-dimensional image data 
model is determined. A 3D model of the product based on the image 
correspondence is rendered. An augmented reality video is generated by 
superimposing the rendered 3D model of the product on the moving model plane 
in the video data. 

Harrington discloses a system for interacting with a real world item such 
as a piece of paper with a virtual display of imagery and/or text to give the illusion 
of holding a printed version of an electronic document. A blank piece of paper is 
disposed within a filed of view and control of a user so that manipulation of the 
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real item is intended to be communicative of user instructions. The item is 
selectively positionable by the user for purposes of communicating certain 
instructions to an applications program controlling the display. 

Unlike the present invention, Harrington deals with the 3D manipulation of 
a 2D object. As such, the positioning of the paper on the screen is an issue as 
well as its depth (whether type is larger or smaller). However Harrington, does 
not need to concern itself with the pose of the object (i.e., the 3D spatial position 
of the object). The present invention is directed to the creation of an augmented 
reality video in which a 3D image of an object is augmented into a video stream. 
Because the object itself is a 3D object, the pose of the object (i.e., the direction 
and angle of the object) is critical in creating an image model that reflects the 
object from the desired visual perspective. This problem is neither recognized 
nor addressed by Harrington since Harrington is concerned with a 2D object. 
Applicants have amended independent claims 46 and 54 to recite that the step of 
"creating a three dimensional image data model of a product in a pose 
corresponding to the pose of the moving model plane". Applicants respectfully 
submit that Harrington does not perform this step since Harrington is not 
concerned with the pose of an object. As such, Applicants submit that Harrington 
does not teach or disclose Applicants' invention as recited in independent claims 
46 and 54. Claims 48, 52, 53, 56 and 60, being dependent upon independent 
claims 46 and 54, are also not taught or disclosed by Harrington. Applicants 
respectfully request that the rejection of claims 46, 48, 52-54, 56 and 60 under 35 
U.S.C. § 102 (e) be withdrawn. 

Rejection of Claims 50, 51, 58, 59, 61 and 62 under 35 U.S.C. § 103 (a) 

The Examiner has rejected claims 50, 51 , 58, 59, 61 and 62 under 35 
U.S.C. § 103 (a) as being unpatentable over Harrington in view of U.S. Patent 
No. 7,050,603 (Rhoads). The Examiner correctly notes that Harrington does not 
teach a three dimensional image data model that represents an animation. The 
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Examiner contends that Rhoads discloses watermarks that can be embedded in 
two dimensional image renderings or animated 3D graphical objects and that the 
embedded object can be composited with a video stream to form a video 
program. The Examiner argues that it would have been obvious to one skilled in 
the art to modify Harrington's system by using projected 3D animated objects as 
disclosed in Rhoads. Applicants respectfully traverse the rejection. 

Rhoads discloses using watermarks in video signals to associate video 
objects in a video sequence with object specific actions or information. 
Applicants respectfully submit that adding a watermark as is disclosed in Rhoads 
to the Harrington system would not teach or disclose Applicants' invention. 
Applicant's invention is directed to the editing of an augmented reality video in 
which a 3D object is augmented into a video stream. The 3D object can be 
displayed from various perspectives and angles so that certain features of the 
object are highlighted. As such, the pose of the 3D object is critical in 
augmented the appropriate view of the 3D object into the video stream. As 
discussed above, Harrington is directed to displaying a paper containing 
information into an electronic image so that user instructions can be displayed. 
Harrington does not teach or disclose a need for displaying a 3D object in 
different poses to highlight different perspectives of the 3D object. It makes even 
less sense for Harrington to disclose an object that is animated. Applicant 
respectfully submits that neither Harrington nor Rhoads, whether taken alone or 
in combination, teaches or disclose Applicant's invention as claimed. Applicant 
requests that the rejection of claims 50, 51 , 58, 59, 61 and 62 under 35 U.S.C. § 
103 (a) be withdrawn. 

Conclusion 

Applicants respectfully submit that claims 46, 48, 50-54 and 56, 58-62, as 
amended, are in condition for allowance and request that a timely Notice of 
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Allowance be issued in this case. The Examiner is invited to contact the 
undersigned should he have any questions in this matter. 



Respectfully submitted, 

^xj/a d- ^ 

Michele L. Conover 
Reg. No. 34,962 
Attorney for Applicant 



Date: January 8, 2006 
Siemens Corporation 
Intellectual Property Department 
1 70 Wood Avenue South 
Iselin, New Jersey 08830 
(732)321-3013 
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